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Uncertainties  in  Projecting  Budget  Surpluses: 
A  Discussion  of  Data  and  Methods 


On  January  3 1 , 200 1 ,  the  Congressional  Budget 
Office  (CBO)  released  The  Budget  and  Eco¬ 
nomic  Outlook:  Fiscal  Years  2002-201 F 
which  presents  CBO’s  latest  projections  of  federal 
revenues  and  outlays  for  that  period.  Chapter  5  of 
that  report  discusses  the  uncertainties  in  CBO’s  base¬ 
line  projection  of  the  total  budget  surplus  and  in¬ 
cludes  a  chart  (reproduced  here  as  Figure  1)  illustrat¬ 
ing  how  those  uncertainties  increase  over  six  years. 
This  supplement  to  that  report  discusses  the  data  and 
methods  used  to  construct  the  chart. 

Figure  1  presents  CBO’s  baseline  projection  of 
the  budget  surplus  as  a  fan  of  probabilities  around  the 
mean  projection  for  fiscal  years  2001  through  2006. 
The  fan  widens  as  the  projection  extends.  The  base¬ 
line  projection  falls  in  the  middle  of  the  highest 
probabilities — the  darkest  part  of  the  figure.  But  the 
figure  makes  clear  that  nearby  projections — other 
paths  in  the  darkest  part  of  the  figure — ^have  nearly 
the  same  probability  as  the  baseline.  Moreover,  pro¬ 
jections  that  are  quite  different  from  the  baseline 
have  a  significant  probability  of  being  realized.^ 

The  uncertainties  shown  in  Figure  1  are  based 
on  CBO’s  historical  record  of  budget  projections. 
That  is,  the  estimates  of  uncertainty  presume  that  in 
the  future,  CBO  will  make  errors  similar  to  those  it 
made  in  the  past,  with  about  the  same  probability  dis¬ 
tribution  of  large  and  small  errors. 

Preparing  Figure  1  involved  two  stages.  In  the 
first  stage,  CBO  constructed  measures  of  its  past  pro¬ 
jection  errors  that  take  into  account  the  effects  of  leg- 


1 .  Technically,  the  probability  density  is  highest  near  the  baseline  and 
falls  off  for  more  distant  projections. 


islation  and  other  factors.  In  the  second  stage,  CBO 
constructed  probability  distributions  for  those  errors 
at  six  time  horizons,  beginning  with  the  current  fiscal 
year  (the  year  in  which  the  projection  was  made)  and 
covering  the  next  five  years.  Those  distributions  are 
depicted  in  Figure  1 . 


Figure  1 . 

Uncertainty  in  CBO’s  Projections  of  the 
Totai  Budget  Surplus  Under  Current  Policies 
(By  fiscal  year) 


1  Jilin:!  *- 


SOURCE:  Congressional  Budget  Office. 

NOTES:  The  figure  shows  the  estimated  likelihood  of  alternative 
projections  of  the  surplus  under  current  policies.  The 
calculations  are  based  on  CBO’s  past  track  record. 
CBO’s  baseline  projection  falls  in  the  middle  of  the 
darkest  area.  Assuming  that  policies  do  not  change, 
the  probability  is  10  percent  that  actual  surpluses  will 
fall  in  the  darkest  area  and  90  percent  that  they  will  fall 
within  the  whole  shaded  area. 

Actual  surpluses  will  of  course  be  affected  by  legisla¬ 
tion  enacted  during  the  next  10  years,  including  deci¬ 
sions  about  discretionary  spending.  The  effects  of  fu¬ 
ture  legislation  are  not  included  in  this  figure. 
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Stage  One:  Constructing 
the  Measures  of  Error 

CBO  made  several  adjustments  to  its  previously  pub¬ 
lished  projections  before  subtracting  them  from  ac¬ 
tual  values  to  determine  past  projection  errors.  The 
projections  of  revenues  and  outlays  were  adjusted  to 
account  for  the  estimated  effects  of  legislation  en¬ 
acted  after  the  projections  were  prepared.  CBO  made 
that  adjustment  because  its  baseline  projections  are 
intended  to  show  the  expected  level  of  the  budget 
deficit  or  surplus  assuming  that  current  tax  and 
spending  policies  continue.  Without  that  adjustment, 
the  measure  of  projection  error  would  include  the 
effects  of  later  legislation,  which  would  run  counter 
to  the  purpose  of  the  baseline. 

In  calculating  the  projection  errors,  CBO  ex¬ 
cluded  discretionary  spending  and  net  interest  pay¬ 
ments  from  both  the  baseline  projections  and  actual 
outlays.  Discretionary  spending  was  omitted  because 
all  discrepancies  between  actual  discretionary  spend¬ 
ing  and  baseline  projections  of  discretionary  spend¬ 
ing  were  counted  as  resulting  from  legislation  (since 
levels  of  discretionary  spending  are  determined  anew 
each  year  through  appropriation  legislation).^  Inter¬ 
est  payments  were  excluded  because  those  payments 
depend  on  the  size  of  the  government’s  publicly  held 
debt —  the  cumulation  of  annual  budget  deficits — so 
errors  in  projecting  interest  payments  depend  on  the 
cumulation  of  errors  in  projecting  the  deficit.  In 
stage  two,  when  CBO  constructed  its  probability  dis¬ 
tributions,  interest  payments  were  brought  back  into 
the  calculations. 

Errors  were  calculated  for  each  year  covered  by 
the  winter  baseline  projections  that  CBO  published 
from  1981  through  2000.  In  most  years,  those  projec¬ 


2.  In  previous  analyses  of  its  track  record,  CBO  split  discrepancies  in 
discretionary  spending  into  three  components;  the  lion’s  share  was 
attributed  to  legislation,  but  small  portions  were  attributed  to  eco¬ 
nomic  and  technical  assumptions.  Attributing  all  discrepancies  in 
discretionary  spending  to  legislation,  as  is  done  here,  permits  the 
use  of  a  larger  historical  record.  Since  1986,  the  Balanced  Budget 
and  Emergency  Deficit  Control  Act  has  mandated  that  the  baseline 
for  discretionary  spending  reflect  assumptions  about  inflation.  As 
a  result,  baselines  for  discretionary  spending  made  before  1986  are 
not  eomparablc  with  those  made  afterward.  Counting  all  discrep¬ 
ancies  in  discretionary  spending  as  legislative  avoids  that  problem. 


tions  were  published  in  January  or  February,  al¬ 
though  in  1996,  publication  was  delayed  until  May. 
Also,  CBO  used  its  July  1981  projection  in  place  of 
the  one  published  in  February  1982.^  The  resulting 
sample  size  was  small:  only  19  for  current-year  pro¬ 
jections,  declining  to  14  for  five-year-ahead  projec¬ 
tions.''  (The  sample  size  diminishes  because  projec¬ 
tions  made  in  the  past  five  years  can  be  compared 
with  actual  data  only  through  2000.) 

The  estimated  effects  of  legislation  were  taken 
primarily  from  data  published  in  CBO’s  twice-yearly 
reports  on  the  budget  and  economic  outlook.  Most  of 
those  reports  show  the  multiyear  budgetary  effects  of 
legislation  enacted  since  the  previous  projection.  For 
cases  in  which  estimates  were  not  available  (as  will 
be  discussed  below),  substitutes  were  constructed. 


Revenues 

The  effects  of  tax  legislation  were  originally  esti¬ 
mated  by  either  the  Congress’s  Joint  Committee  on 
Taxation  (JCT)  or  CBO  at  the  time  the  legislation 
was  being  considered.  Those  estimates  are  not  re¬ 
vised  after  their  initial  publication,  even  though  later 
economic  and  technical  information  might  permit 
better  estimates  (for  example,  knowledge  of  an  actual 
tax  base,  such  as  wages  or  corporate  profits,  would 
improve  estimates  of  how  a  change  in  tax  law  would 
affect  revenues).  Using  unrevised  data  on  the  effects 
of  legislation  may  overstate  the  true  uncertainty  of 
CBO’s  budget  projections,  other  things  held  equal. 

Those  estimated  effects  of  tax  legislation  were 
used  to  adjust  each  baseline  revenue  projection.  For 
example,  the  baseline  projection  of  revenues  made  in 
January  1994  for  fiscal  year  1999  was  adjusted  down¬ 
ward  from  $1,630  billion  to  $1,619  billion  (see  Table 
1).  That  adjustment  reflected  all  tax  legislation  en¬ 
acted  after  the  January  1994  projection  was  published. 


3.  Specifically,  CBO  docs  not  have  enough  information  in  its  files  to 
include  the  estimated  effects  of  legislation  enacted  between  Febru¬ 
ary  1982  and  February  1983.  Much  better  data  arc  available  for 
the  slightly  longer  period  of  July  1981  through  February  1983. 

4.  The  sample  size  could  have  been  doubled  by  including  the  updated 
projections  that  CBO  typically  publishes  in  the  summer,  but  those 
updates  arc  closely  related  to  the  winter  baselines  and  do  not  really 
offer  additional  information. 
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Table  1. 

How  CBO’s  January  1994  Revenue  Projection  Was  Adjusted  for  Subsequent  Legislation 
(By  fiscal  year,  in  billions  of  dollars) 


1994 

1995 

1996 

1997 

1998 

1999 

Baseline  Revenue  Projection 

1,251 

1,338 

1,411 

1,479 

1,556 

1,630 

Subsequent  Legislation 

0 

0 

January  1994  to  August  1994 

0 

0 

0 

0 

August  1994  to  January  1995 

0 

1 

-1 

-1 

-3 

-3 

January  1995  to  August  1995 

* 

* 

August  1995  to  May  1996 

0 

* 

May  1996  to  August  1996 

-1 

-3 

-2 

-2 

August  1996  to  January  1997 

Jr 

1 

January  1997  to  September  1997 

2 

-10 

-7 

September  1997  to  January  1998 

1 

January  1998  to  August  1998 

1 

August  1998  to  January  1999 

0 

January  1999  to  July  1999 

July  1999  to  January  2000 

0 

Total 

0 

* 

-2 

-1 

-14 

-11 

Adjusted  Baseline  Projection 

1,251 

1,338 

1,409 

1,478 

1,542 

1,619 

SOURCE:  Congressional  Budget  Office. 

NOTES:  The  only  major  changes  in  tax  law  enacted  after  CBO’s  January  1994  baseline  projection  were  made  in  the  Taxpayer  Relief  Act  of 

1997.  Its  effects  were  incorporated  into  CBO’s  September  1997  baseline  projection.  Two  other  adjustments  are  notable  but  relatively 
minor.  The  January  1995  baseline  reflected  various  reductions  in  tariff  rates  but  primarily  those  in  the  Generalized  System  of 

Preferences.  Similarly,  the  downward  adjustment  in  the  August  1996  baseline  projection  reflected  two  bills: 

the  Health  Insurance 

Portability  Act  of  1996  (H.R.  3103)  and  the  Small  Business  Job  Protection  Act  of  1996  (H.R.  3448). 

*  =  between  -$500  million  and  $500  million. 

Much  of  that  legislation  was  contained  in  the  Tax¬ 
payer  Relief  Act  of  1997.  CBO  and  JCT  estimated 
that  the  provisions  in  that  act  would  reduce  revenues 
by  $7  billion  in  fiscal  year  1999.^  Similar  adjust¬ 
ments  were  made  for  the  other  years  in  the  baseline 
projections.  The  differences  between  those  adjusted 
baseline  projections  and  actual  revenues  represent  the 
errors  attributable  to  economic  and  technical  factors 
(see  Table  2). 


5.  That  $7  billion  reduction  was  the  net  effect  of  a  $29  billion  de¬ 
crease  and  a  $22  billion  increase  ($12  billion  of  which  reflected 
shifts  in  the  timing  of  tax  collections).  Sec  Congressional  Budget 
Office,  The  Economic  and  Budget  Outlook:  An  Update  (September 
1997),  p.  36. 


Outlays 

The  estimated  effects  of  legislation  on  outlays  (ex¬ 
cluding  discretionary  spending  and  interest  pay¬ 
ments)  were  also  taken  largely  from  CBO’s  reports 
on  the  budget  and  economic  outlook.  However,  some 
adjustment  to  that  information  was  necessary. 

o  Baseline  Projections  of  Discretionary  Spend¬ 
ing.  As  noted  above,  differences  between  actual 
and  projected  levels  of  discretionary  spending 
were  assumed  to  be  attributable  to  legislation. 
But  the  July  1981  projection  did  not  include  a 
separate  category  for  discretionary  spending. 
For  that  baseline  only,  discretionary  spending 
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Table  2. 

Errors  in  CBO’s  Baseline  Projections  of  Revenues  (As  a  percentage  of  GDP) 


Date  the  Projection  _ Fiscal  Year  for  Which  the  Projection  Was  Made 


Was  Published 

Current  Year 

Yearl 

Year  2 

Year  3 

Year  4 

Years 

July  1981 

-0.4 

-1.6 

-3.9 

-3.8 

-4.1 

-5.0 

February  1983 

-0.2 

0.2 

* 

-0.6 

-0.4 

-0.7 

February  1984 

0.1 

-0.2 

-1.1 

-1.6 

-1.3 

February  1985 

* 

-0.5 

-0.4 

-0.9 

-0.6 

-1.5 

February  1986 

-0.2 

-0.2 

-0.6 

-0.3 

-1.1 

-2.3 

January  1987 

0.4 

★ 

-0.7 

-2.0 

-2.7 

February  1988 

0.2 

0.7 

-0.1 

-1.3 

-1.8 

-2.2 

January  1989 

0.1 

-0.6 

-1.7 

-2.2 

-2.4 

-2.3 

January  1990 

-0.6 

-1.7 

-2.1 

-2.3 

-2.3 

-2.3 

January  1991 

-0.6 

-1.1 

-1.5 

-1.4 

-1.5 

-1.2 

January  1992 

0.1 

-0.4 

-0.4 

-0.1 

0.4 

January  1993 

0.2 

0.2 

0.2 

0.6 

1.3 

2.2 

January  1994 

0.1 

0.2 

0.6 

1.2 

2.1 

2.3 

January  1995 

* 

0.5 

1.3 

2.2 

2.4 

3.5 

May  1996 

0.3 

1.1 

2.2 

2.4 

3.7 

n.a. 

January  1997 

0.9 

1.9 

2.2 

3.4 

n.a. 

n.a. 

January  1998 

0.6 

1.1 

2.4 

n.a. 

n.a. 

n.a. 

January  1999 

0.1 

1.5 

n.a. 

n.a. 

n.a. 

n.a. 

January  2000 

0.8 

n.a. 

n.a. 

n.a. 

n.a. 

n.a. 

SOURCE:  Congressional  Budget  Office. 

NOTES:  Errors  are  actual  revenues  minus  projected  revenues  as  a  percentage  of  gross  domestic  product. 


*  =  between  -0.05  and  0.05  percent;  n.a.  =  not  applicable. 


was  approximated  by  adding  the  projections  for 
defense,  other  grants  to  state  and  local  govern¬ 
ments,  and  other  federal  operations.^ 

0  Insufficient  Details  About  Legislation.  In 

some  cases,  the  estimated  effects  of  legislation  were 
not  published  in  enough  detail  to  exclude  the  ef¬ 
fects  of  legislation  on  discretionary  spending.  In 
other  cases,  the  data  were  published  for  some  but 
not  all  of  the  six  years  in  the  baseline  budget  pro¬ 
jection.  One  or  both  of  those  problems  applied 
to  the  following  periods:  August  1986  to  January 
1987,  August  1987  to  February  1988,  August 
1994  to  January  1995,  and  January  1998  to  Au- 


6.  See  Congressional  Budget  Office,  Baseline  Budget  Projections: 
Fiscal  Years  1982-1986  (My  1981),  p.  38, 


gust  1998.  In  those  cases,  supplemental  infor¬ 
mation  from  CBO’s  files  was  used  to  estimate 
the  needed  numbers. 

As  with  revenues,  the  estimated  effects  of  legis¬ 
lation  on  outlays  were  used  to  adjust  each  baseline 
projection  of  outlays  (after  removing  the  projections 
of  discretionary  spending  and  interest  payments). 
The  differences  between  those  adjusted  projections 
and  actual  outlays  are  the  errors  attributable  to  eco¬ 
nomic  and  technical  factors  (see  Table  3). 


Primary  Budget  Surplus  or  Deficit 

The  primary  budget  surplus — or  deficit  in  earlier 
years — excludes  net  interest  payments.  The  errors  for 
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Table  3. 

Errors  in  CBO’s  Baseline  Projections  of  Outlays  (As  a  percentage  of  GDP) 


Date  the  Projection 

Was  Published 

Fiscal  Year  for  Which  the  Projection  Was  Made 

Current  Year 

Yearl 

Year  2 

Year  3 

Year  4 

Years 

July  1981 

-0.5 

-0.3 

-0.1 

-0.7 

-0.6 

-0.6 

February  1983 

-0.2 

-0.4 

-0.1 

* 

* 

* 

February  1984 

-0.1 

* 

* 

-0.1 

-0.1 

-0.3 

February  1985 

0.1 

0.2 

0.1 

0.2 

0.1 

1.4 

February  1986 

0.3 

0.3 

0.3 

0.2 

1.5 

1.5 

January  1987 

-0.2 

0.1 

-0.1 

1.1 

1.1 

1.3 

February  1988 

0.1 

-0.1 

1.0 

1.0 

1.2 

0.8 

January  1989 

-0.2 

1.0 

0.9 

1.1 

0.7 

0.9 

January  1990 

0.8 

0.7 

0.8 

0.4 

0.7 

0.4 

January  1991 

-1.3 

-1.3 

-0.7 

-0.2 

0.6 

0.5 

January  1992 

-1.0 

-1.4 

-0.6 

-0.2 

0.2 

-0.4 

January  1993 

-0.6 

-0.5 

-0.8 

-0.5 

-0.7 

-0.8 

January  1994 

-0.2 

-0.3 

-0.3 

-0.7 

-0.8 

-1.0 

January  1995 

-0.2 

-0.4 

-0.8 

-0.9 

-1.0 

-1.1 

May  1996 

-0.2 

-0.5 

-0.8 

-0.8 

-0.9 

n.a. 

January  1997 

-0.3 

-0.4 

-0.6 

-0.6 

n.a. 

n.a. 

January  1998 

-0.1 

-0.3 

-0.4 

n.a. 

n.a. 

n.a. 

January  1999 

★ 

-0.2 

n.a. 

n.a. 

n.a. 

n.a. 

January  2000 

* 

n.a. 

n.a. 

n.a. 

n.a. 

n.a. 

SOURCE:  Congressional  Budget  Office. 

NOTES:  Errors  are  actual  outlays  minus  projected  outlays  as  a  percentage  of  gross  domestic  product.  They  exclude  errors  In  the  baseline 
projections  of  discretionary  spending  (which  are  assumed  to  be  attributable  solely  to  legislation)  and  errors  in  the  baseline  projections 
of  net  interest  (which  depend  on  the  errors  in  the  surplus  excluding  interest). 

*  =  between  -0.05  and  0.05  percent;  n.a.  =  not  applicable. 


outlays  were  subtracted  from  the  errors  for  revenues 
to  determine  CBO’s  errors  in  projecting  the  primary 
budget  surplus  adjusted  for  the  effects  of  tax  and 
spending  legislation  (see  Table  4).  Those  errors  were 
cumulated  into  errors  in  projecting  publicly  held  debt, 
which  in  turn  were  used  in  stage  two  to  estimate  the 
uncertainty  of  CBO’s  projections  of  interest  payments. 


Stage  Two:  Constructing 
Probability  Distributions 

Drawing  reliable  inferences  about  the  uncertainty  of 
CBO’s  budget  projections  over  a  six-year  horizon  on 
the  basis  of  only  14  observations  requires  making 


some  important  assumptions.  This  section  describes 
the  basic  assumptions  that  allowed  CBO  to  construct 
probability  distributions  of  its  projections  of  the  ad¬ 
justed  primary  surplus  based  on  its  track  record,  as 
well  as  the  assumptions  that  CBO  used  to  calculate 
uncertainty  in  its  projections  of  the  total  surplus  (the 
primary  surplus  plus  net  interest). 

The  Probability  Model 

The  uncertainty  calculations  are  based  on  a  simple 
probability  model  for  the  primary  surplus.  For  the 
current-year  projections,  errors  are  assumed  to  be 
normally  distributed  with  a  mean  of  zero.  For  subse¬ 
quent  years,  projection  errors  are  assumed  to  be  cu¬ 
mulative:  that  is,  if  a  projection  misses  early  on,  the 
error  is  likely  to  be  carried  into  subsequent  years. 
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Table  4. 

Errors  in  CBO’s  Projections  of  the  Adjusted  Primary  Surplus 


Date  the  Projection  _ Fiscal  Year  for  Which  the  Projection  was  Made _ 

Was  Published  Current  Year  Year  1  Year  2  Year  3  Year  4  Year  5 


Errors  as  a  Percentage  of  Actual  Revenues 


July  1981 

0.3 

-6.9 

-21.3 

-18.1 

-19.6 

-25.0 

February  1983 

* 

3.3 

1.1 

-3.3 

-2.1 

-4.0 

February  1 984 

1.2 

-1.3 

-5.5 

-5.3 

-7.9 

-5.6 

February  1 985 

-0.4 

-4.1 

-2.9 

-5.7 

-3.5 

-15.8 

February  1986 

-3.2 

-2.7 

-5.3 

-2.8 

-14.5 

-21.7 

January  1987 

3.5 

-1.0 

1.7 

-10.2 

-17.9 

-22.5 

February  1 988 

0.7 

4.3 

-6.3 

-13.2 

-17.2 

-16.9 

January  1989 

1.9 

-9.2 

-14.7 

-18.7 

-17.4 

-18.1 

January  1990 

-7.8 

-13.3 

-17.0 

-15.8 

-16.2 

-14.4 

January  1991 

3.5 

1.4 

-4.4 

-6.8 

-11.1 

-8.9 

January  1992 

6.1 

5.7 

1.2 

-1.5 

-1.8 

3.9 

January  1993 

4.3 

4.4 

5.6 

6.1 

10.2 

15.3 

January  1994 

1.8 

2.4 

4.4 

10.1 

14.7 

16.2 

January  1995 

0.8 

4.8 

10.6 

15.3 

16.9 

22.5 

May  1996 

2.7 

8.6 

14.8 

16.4 

22.2 

n.a. 

January  1997 

6.2 

11.5 

13.7 

19.6 

n.a. 

n.a. 

January  1998 

3.9 

6.6 

13.9 

n.a. 

n.a. 

n.a. 

January  1999 

0.8 

8.3 

n.a. 

n.a. 

n.a. 

n.a. 

January  2000 

4.1 

n.a. 

n.a. 

n.a. 

n.a. 

n.a. 

Mean  Error 

1.6 

1.3 

-0.6 

-2.1 

-4.4 

-6.8 

Root  Mean  Square  Error 

3.6 

6.5 

10.4 

12.1 

14.3 

16.5 

(Continued) 


Because  the  sample  of  current-year  errors  is  too 
small  to  estimate  a  probability  distribution  with  any 
confidence,  CBO  assumed  that  the  errors  were  gener¬ 
ated  by  a  normal  distribution.  An  appealing  aspect  of 
that  assumption  is  that  the  entire  distribution  is  deter¬ 
mined  by  specifying  only  two  parameters:  the  mean 
and  the  standard  deviation.  Although  CBO’s  mean 
error  in  projecting  the  primary  surplus  since  1981 
was  1.6  percent  of  actual  revenues  (see  Table  4), 
CBO  assumed  that  the  mean  error  was  zero.’  That 
seems  like  a  reasonable  assumption  since  CBO  aims 
to  produce  unbiased  forecasts. 

The  standard  deviation,  or  dispersion,  of  the 
distribution  of  the  current-year  projection  errors  was 
assumed  to  equal  the  root  mean  square  of  the  errors 

7.  The  sample  mean  was  just  barely  statistically  significant  at  the 
5  percent  confidence  level. 


(RMSE)  in  the  track  record.®  A  normal  distribution 
with  that  standard  deviation  produces  a  very  simpli¬ 
fied  version  of  the  actual  distribution  of  errors  in 
CBO’s  current-year  projections  (see  Figure  2).  The 
actual  errors  seem  very  spiky  compared  with  the  as¬ 
sumed  distribution  because,  in  a  small  sample,  indi¬ 
vidual  events  loom  large. 

The  probability  model  for  subsequent  years  takes 
into  account  the  cumulative  nature  of  the  errors.  In 
CBO’s  track  record,  the  projection  errors  for  the  ad¬ 
justed  primary  siuplus  are  correlated  over  the  projec¬ 
tion  horizon,  as  would  be  expected.  The  correlation 

8.  The  root  mean  square  error  is  calculated  by  squaring  each  projec¬ 
tion  error,  averaging  the  squares,  and  taking  the  square  root  of  the 
result.  When  the  errors  are  known  to  have  a  population  mean  of 
zero,  the  RMSE  is  an  unbiased  estimate  of  the  population  standard 
deviation.  When  the  population’s  true  mean  is  not  zero,  the  RMSE 
would  be  expected  to  exceed  the  standard  deviation. 
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Table  4. 
Continued 


Date  the  Projection  _ Fiscal  Year  for  Which  the  Projection  was  Made _ 

Was  Published  Current  Year  Year1  Year  2  Year  3  Year  4  Year  5 


Changes  in  Errors  (Percentage  points) 


July  1981 
February  1983 
February  1984 
February  1985 
February  1986 
January  1987 
February  1988 
January  1989 
January  1990 
January  1991 
January  1992 
January  1993 
January  1994 
January  1995 
May  1996 
January  1997 
January  1998 
January  1999 

Mean  Error 

Root  Mean  Square  Error 


n.a. 

-7.2 

-14.5 

n.a. 

3.3 

-2.3 

n.a. 

-2.5 

-4.3 

n.a. 

-3.7 

1.1 

n.a. 

0.5 

-2.6 

n.a. 

-4.5 

2.6 

n.a. 

3.6 

-10.6 

n.a. 

-11.1 

-5.5 

n.a. 

-5.5 

-3.7 

n.a. 

-2.2 

-5.8 

n.a. 

-0.4 

-4.5 

n.a. 

0.1 

1.2 

n.a. 

0.6 

2.1 

n.a. 

4.0 

5.8 

n.a. 

5.9 

6.2 

n.a. 

5.2 

2.3 

n.a. 

2.7 

7.2 

n.a. 

7.5 

n.a. 

n.a. 

-0.2 

-1.5 

n.a. 

4.8 

5.9 

3.2 

-1.5 

-5.4 

-4.4 

1.2 

-1.8 

0.2 

-2.6 

2.4 

-2.7 

2.2 

-12.3 

2.5 

-11.7 

-7.2 

-11.8 

-7.8 

-4.6 

-6.9 

-4.0 

0.3 

-4.0 

1.3 

-0.7 

1.2 

-0.5 

1.8 

-2.4 

-4.3 

2.2 

-2.6 

-0.3 

5.7 

0.5 

4.1 

5.1 

5.6 

4.6 

1.5 

4.7 

1.6 

5.6 

1.6 

5.8 

n.a. 

5.9 

n.a. 

n.a. 

n.a. 

n.a. 

n.a. 

n.a. 

n.a. 

n.a. 

-0.6 

-0.8 

-0.5 

4.7 

4.6 

5.1 

SOURCE;  Congressional  Budget  Office. 

NOTES;  Errors  are  actual  surpluses  minus  projected  surpluses  as  a  percentage  of  actual  revenues.  Relating  the  errors  to  federal  revenues 
rather  than  to  gross  domestic  product  (GDP)  renders  the  error  analysis  independent  of  CBO’s  economic  projections  as  well  as  of  the 
periodic  revisions  to  GDP  that  complicate  retrospective  analyses  of  CBO  projections  of  national  income  variables.  For  a  given  set  of 
historical  estimates,  the  ratio  of  revenues  to  GDP  is  fairly  stable.  Thus,  substituting  revenues  for  GDP  in  this  error  analysis  is  not 
likely  to  distort  the  conclusions. 

The  mean  error  is  the  average  of  the  errors.  The  root  mean  square  error  is  the  square  root  of  the  average  of  the  squared  errors. 

*  =  between  -0.05  and  0.05  percent;  n.a.  =  not  applicable. 


between  the  errors  for  the  current-year  and  first-year- 
ahead  projections  is  about  0.7.  For  more  distant  pro¬ 
jections  it  is  even  higher,  reaching  almost  1  between 
the  fourth-  and  fifth-year-ahead  projections.  Those 
correlations,  however,  do  not  indicate  how  much  the 
influence  of  one  error  spreads  to  subsequent  errors 
one  or  more  years  ahead  in  the  same  projection. 

To  assess  the  degree  to  which  the  errors  grow 
over  the  projection  horizon,  CBO  estimated  regression 
equations  in  which  each  period’s  projection  error  was 


specified  as  some  multiple  of  the  previous  period’s 
error.  To  illustrate  CBO’s  method,  assume  that  the 
symbol  Ut(h)  denotes  the  projection  error  made  for  the 
primary  surplus  in  year  t+h  as  projected  in  year  t,  with 
h  taking  the  value  of  any  integer  between  zero  and  5. 
(For  example,  the  notation  Ui99o(3)  would  be  the  error 
CBO  made  in  projecting  the  1993  primary  surplus  in 
the  projection  it  published  in  January  1990.  That  error 
equaled  -15.8  percent  of  revenues,  as  shown  in  Table 
4.)  The  error  U^h)  is  assumed  to  evolve  through  the 
projection  horizon  according  to  the  following  rule: 
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Figure  3. 

Frequency  Distributions  for  Changes  in  Errors 
in  Projecting  the  Current-Year  Primary  Surplus 


UXh)  =  A,U,(h-l)  +  e,(h) 

in  which  E^(h)  is  a  random  shock  and  is  a  constant 
propagation  parameter  for  each  value  of  h  =  1, . . . ,  5. 
CBO  ran  five  regressions  yielding  estimates  of  Aj, 
A2,  A3,  A4,  and  A5,  In  each  case,  the  estimated  propa¬ 
gation  parameter  slightly  exceeded  1 .  However,  in 
only  one  case  (h  =  2)  did  the  coefficient  exceed  1  to  a 
degree  that  was  statistically  significant  at  the  5  per¬ 
cent  confidence  level. 

For  simplicity,  CBO  assumed  that  each  error 
was  equal  to  the  previous  year’s  error  plus  a  random  shock 
— ^that  is,  CBO  assumed  A^  =  1  for  each  horizon  h.  In 
light  of  the  very  limited  data  available,  CBO  also  as¬ 
sumed  that  changes  in  the  projection  errors  at  different 
horizons  (the  efii),  when  A^  =  1)  were  drawn  from  a 
common  distribution.  Specifically,  CBO  assumed  that 
the  £t(h)  are  normally  distributed  with  a  mean  of  zero 
and  a  standard  deviation  equal  to  the  root  mean  square 
of  the  change  in  CBO’s  errors  for  all  values  of  t  and  h. 
That  normal  distribution  described  the  pooled  St(h) 


Figure  2. 

Frequency  Distributions  for  Errors  in  Projecting 
the  Current-Year  Primary  Surplus 

Percentage  of  All  Errors 

30  I - 1 


-15  -12  -9  -6  -3  0  3  6  9  12  15 

Errors  as  a  Percentage  of  Actual  Revenues 


SOURCE:  Congressional  Budget  Office. 

a.  Calculated  by  evaluating  the  normal  probability  density  function 
with  a  mean  of  zero  and  a  standard  deviation  equal  to  3.6  (the 
root  mean  square  error  of  CBO’s  errors  in  projecting  the 
current-year  primary  surplus). 


Percentage  of  All  Changes 


-21  -18  -15-12  -9  -6  -3  0  3  6  9  12  15  18  21 


Changes  In  Errors  (Percentage  points) 


SOURCE:  Congressional  Budget  Office. 

a.  Calculated  by  evaluating  the  normal  probability  density  function 
with  a  mean  of  zero  and  a  standard  deviation  equal  to  5.1  (the 
root  mean  square  en'or  of  the  changes  in  CBO’s  errors  in  pro¬ 
jecting  the  primary  surplus). 


reasonably  well  (see  Figure  3).  The  spikiness  of  the 
actual  data  is  a  little  less  pronounced  than  in  Figure  2 
because  the  data  for  different  projection  horizons  are 
grouped  together,  effectively  increasing  the  sample 
size  and  reducing  the  influence  of  particular  unusual 
events. 

By  assuming  Ah  =  I  when  the  numerical  regres¬ 
sion  estimates  exceed  1,  CBO’s  estimates  of  the  un¬ 
certainty  in  the  primary  surplus  are  tilted  to  the  low 
side.  If  in  fact  CBO  had  assumed  that  the  propaga¬ 
tion  parameters  exceed  1,  the  estimated  uncertainty 
of  CBO’s  five-year-ahead  projection  errors  would  be 
considerably  larger  than  reported  here. 


Calculating  the  Uncertainty 
of  Projections 

CBO’s  uncertainty  calculations  were  based  on  simula¬ 
tions  of  the  probability  model  for  the  primary  surplus. 
CBO  performed  8,000  simulations;  in  each  one,  an 
error  was  chosen  at  random  from  the  distribution  for 
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Tables. 

Estimated  Probability  Distribution  of  Projections  of  the  Surplus  (By  fiscal  year,  in  billions  of  dollars) 


Percentile 

2001 

2002 

2003 

2004 

2005 

2006 

5th 

150 

69 

26 

-21 

-76 

-92 

10th 

178 

125 

102 

74 

40 

35 

15th 

198 

162 

149 

130 

108 

125 

20th 

214 

191 

188 

179 

166 

196 

25th 

228 

214 

220 

220 

220 

255 

30th 

240 

237 

250 

260 

266 

308 

35th 

251 

258 

278 

295 

308 

361 

40th 

261 

276 

304 

330 

350 

406 

45th 

271 

293 

330 

364 

387 

454 

50th 

281 

311 

356 

396 

428 

503 

55th 

290 

329 

383 

427 

470 

547 

60th 

301 

348 

409 

461 

507 

593 

65th 

311 

367 

436 

495 

549 

643 

70th 

321 

387 

464 

531 

594 

696 

75th 

333 

409 

495 

570 

642 

749 

80th 

346 

435 

530 

611 

694 

808 

85th 

363 

464 

571 

661 

756 

881 

90th 

383 

497 

620 

722 

834 

987 

95th 

412 

556 

695 

820 

951 

1,131 

SOURCE:  Congressional  Budget  Office. 

NOTE:  These  numbers — constructed  using  8,000  simulations  of  a  simple  probability  model  based  on  CBO’s  track  record — are  the  estimated 
data  that  underlie  the  fan  chart  presented  as  Figure  5-1  in  The  Budget  and  Economic  Outlook:  Fiscal  Years  2002-2011,  p.  94.  The 
row  in  the  table  corresponding  to  the  50th  percentile  is  CBO’s  baseline  projection  of  the  surplus.  These  estimates  permit  the  construc¬ 
tion  of  probability  statements  about  CBO’s  baseline  projection.  For  example,  the  table  indicates  that  there  is  a  90  percent  chance  that 
the  budget’s  balance  in  2006  will  be  between  -$92  billion  (a  deficit)  and  $1,131  billion,  and  a  50  percent  chance  that  the  surplus  in 
2006  will  be  within  about  $245  billion  of  the  baseline  projection.  (That  last  calculation  takes  the  range  from  the  25th  to  the  75th 
percentiles  and  halves  it.) 


current-year  errors,  and  five  additional  errors  were 
chosen  at  random  from  the  distribution  for  the 
changes  in  errors.  Each  of  those  simulations  then 
provided  a  possible  path  of  errors  for  the  primary 
surplus  over  a  six-year  period. 

The  total  surplus  shown  in  the  fan  chart  (Figure 
1)  also  requires  information  about  how  the  predicted 
errors  in  the  primary  budget  (the  budget  excluding 
net  interest)  will  affect  the  government’s  debt-service 
costs.  CBO  ran  each  of  the  8,000  simulations  of  the 
primary  budget  through  a  simple  debt-service  model. 
For  each  period,  that  model  tracks  how  the  errors  in 
the  surplus  projection  translate  into  errors  in  debt, 
applying  an  interest  rate  that  is  a  weighted  average  of 
CBO’s  current  baseline  projections  of  rates  on  three- 
month  Treasury  bills  and  10-year  Treasury  notes. 


That  model  is  an  approximation  of  the  model  CBO 
uses  for  its  budget  projections.^ 

The  result  is  a  collection  of  8,000  simulated  er¬ 
ror  paths,  which  were  added  to  CBO’s  baseline  pro¬ 
jections  to  create  8,000  simulated  projections  of  the 
total  surplus.  The  percentile  distribution  of  those 
projections  (calculated  independently  for  each  pro¬ 
jection  horizon),  which  is  shown  in  Table  5,  was  used 
to  create  the  fan  chart. 


9.  CBO  also  simulated  the  projection  errors  under  several  alternative 
assumptions  about  the  uncertainty  in  interest  rates.  Including  that 
additional  uncertainty,  however,  did  not  appreciably  affect  the 
results. 
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Conclusions 

Table  5  and  the  chart  derived  from  it  (Figure  1)  serve 
to  illustrate  the  uncertainty  of  CBO’s  budget  projec¬ 
tions.  The  calculations  for  that  chart,  however,  are 


based  on  a  very  limited  set  of  data  and  rest  on  impor¬ 
tant  assumptions.  Thus,  those  calculations  are  them¬ 
selves  uncertain,  and  the  numbers  shown  in  Table  5 
should  not  be  considered  precise.  CBO  would  wel¬ 
come  suggestions  about  how  to  improve  this  analysis. 


